The subject matter described herein relates to signal analysis and applications for enhancing data characterization.
Experimentally acquired data typically includes noise in addition to signals representing information and/or events of interest. The noise represents undesired variations that are not related to the desired data. For example, the acquired data can include stochastic variations generated by interactions with the environment surrounding a measured system or a detector acquiring the data. Noise can be generated within the measured system by events that are unrelated to the information of interest. Noise may also be generated when the acquired data is transmitted or processed, for example, when it is digitized. Noise can be a significant problem with devices employing an array of sensors in which there are numerous sources of signals.
For example, biological molecules can be analyzed by biochips or mass spectrometers. In mass spectroscopy, large molecules in a source sample are ionized, fragmented and transported for mass analysis using electromagnetic fields. The measured mass spectrum is distorted by noise that is generated by variations in the generated ions or fragments, or fluctuations and inhomogeneities of the electromagnetic fields.
Biochips are microarrays of biological detectors (probes) to detect biological materials, such as oligonucleotides, peptides, cDNAs, mRNAs or proteins. High-density microarrays include a large number of probes on a single substrate. For example, a microarray can include hundred to a million spot, where each spot represents a particular type of probe. A spot can include one to a thousand million probe molecules that are complementary to a particular biological material. In a microarray experiment, sample molecules are labeled with fluorescence or other photoactive dyes. The labeled molecules hybridize with the complementary biological detectors in the microarray, and a result of the hybridization is determined by scanning photoactivity in the microarray. The scanned photoactivity is distorted by noise that is generated by defects in the microarray, non-complementary hybridization or resolution of the scanning.
Techniques to detect, measure and process signal in noisy data include traditional “passive” techniques. Passive noise analysis estimates a noise level in the data, and identifies signals that are above the noise level. For periodic signals, noisy data is transformed into a frequency (Fourier) representation in which noise components are estimated at multiple frequencies. Signals are detected at frequencies where the frequency component of the noisy data is larger than the estimated noise component. For non-periodic signals that depend on a time or a space coordinate, the noise level is estimated from temporal or spatial fluctuations, respectively. If the shape of the signal is known, the noisy data can be filtered based on the known shape, and a signal can be detected if the filtered data has a component above the noise level. Passive techniques also include maximum a-posteriori (MAP) techniques, maximum likelihood estimator (MLE) techniques, singular value decomposition (SVD), parametric distributional clustering (PCA), neural networks, fuzzy logic systems and Bayesian inferencing systems.
Some physical systems (hardware-based systems) use “active” signal processing. Active signal processing actively enhances signals that are below the noise level. In these systems objects of interest are detected through their interaction with particular excitations. For example, intelligent radars irradiate a moving object, such as an aircraft, with radar pulses that interact with the object and the reflection is received by an antenna. The object is identified by comparing the received signals against a background that is defined by previously transmitted signals. Another example is a superconducting quantum interferometric device (SQUID), in which coupled superconductive half rings are excited to detect magnetic fields. Other examples include imagery intelligence (IMINT), signals intelligence (SIGINT) and electronic intelligence (ELINT) devices. Active signal processing has also been performed by techniques using femto-lasers. However, while hardware-based signal processing systems allow one to characterize signals that would otherwise be obscured by noise using passive techniques, such systems are costly to manufacture and operate and can require several minutes to make a single measurement. In addition, active signal processing techniques may only be used during an experiment and do not permit the characterization of previously obtained experimental data.
It can therefore be appreciated that a need remains for a technique to characterize signals of interest both above and below noise thresholds that can make rapid measurements of both retrospective and prospective data sets, and that does not require expensive hardware.